RELATIVE DIVERGENCE MEASURES 
AND INFORMATION INEQUALITIES 

INDER JEET TANEJA 

Abstract. There are many information and divergence measures exist 
in the literature on information theory and statistics. The most famous 
among them are Kullback-Leiber's |17| relative information and Jeffreys 
J-divergence, Information radius or Jensen difference divergence mea- 
sure due to Sibson [221 . Burbea and Rao has also found its applications 
in the literature. Taneja |25j studied another kind of divergence measure 
based on arithmetic and geometric means. These three divergence mea- 
sures bear a good relationship among each other. But there are another 
measures arising due to J -divergence, JS-divergence and AG- divergence. 
These measures we call here relative divergence measures or non-symmetric 
divergence measures. Here our aim is to obtain bounds on symmetric and 
non-symmetric divergence measures in terms of relative information of type 
s using properties of Csiszar's f-divergence. 



1. Introduction 



Let 



(pi,P2, -,Pn) 



Pi > 0,^p; = 1 > , n ^ 2, 



»=i 



be the set of all complete finite discrete probability distributions. 
The Kullback Leibler's (1951) relative information is given by 



(1.1) K(P\\Q) = J2pM~ 



for all P, Q e r n . 

We observe that the measure (jl.lj) is not symmetric in P and Q. Its sym- 
metric version famous as J-divergence (Jeffreys ^H]; Kullback and Leiber [T7j ) 
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is given by 



(1.2) J{P\\Q) = K(P\\Q) + K(Q\\P) = (ft - *) ln (-)- 

i=i qi 

For simplicity, we call here the measure K(Q\\P) the adjoint of K(P\\Q) 
and vice-versa. 

Alternatively, the measure J(P\\Q) can also be written in the following way: 

(1.3) J(P\\Q) = D(P\\Q)+D(Q\\P), 
where 



;i-4) D(P\\Q) = f^( Pi - qi )ln(?^j 



and 



(1-5) D{Q\\P) = j2{ qi - Pi )\n{?^y 

Let us consider the following two measures: 

(1.6) F( P||Q) = tf(p||^)=|> ta (.^) 
and 

(1.7) G(P\\Q) = K ( f±«||p) =±(*±2) In ^ + « 



The adjoint forms of the measures (|1.6|) and (|1.7|) are given by 

(1 . 8) FW | W = jr (p,|£+0) = J wln (^_) 

and 

(1.9) gwiip) -*(z±a w )-±( p -4^) ^ fp,+ * 



2 -V trv 2 7 v 2, 

respectively, and the symmetric forms are given by 

(1.10) I{P\\Q) = ±[F(P\\Q) + F{Q\\P)] 

and 

(1.H) T(P\\Q)= 1 -[G(P\\Q) + G(Q\\P)}, 
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respectively. 

The three measures J(P\\Q), I(P\\Q) and T(P\\Q) are related with each 
other as 



(1.12) 



J(P\\Q)=A[I(P\\Q)+T(P\ 



Moreover, the measures (jl.4j) can also be written as 
(1.13) D(P\\Q) = 2[F(Q\\P) + G(Q\\P)} 

The measure I(P\\Q) is famous in the literature as information radius (Sib- 
son [23 ) or Jensen difference divergence measure, or simply, JS-divergence 
(Burbea and Rao [3 II])- The measure (jl.9j) is new in the literature and is 
studied for the first time by Taneja [25j . called arithmetic and geometric mean 
divergence measure or simply, AG- divergence. More details on these divergence 
measures can be seen in Taneja |2*4*| l2*o]. 

For simplicity, we shall call the measure D(P\\Q) by relative J-divergence, 
the measure F(P\\Q) by relative JS-divergence and the measure G(P\\Q) by 
relative AG- divergence. The measure D(P\\Q) is due to Dragomir et al. [T4*] . 
The measure F(P\\Q) has been studied by many authors (Shioya and Da-te 
[22]; Barnet et al. [T] ; Lin [E]; Lin and Wong 20J. The measure G(P\\Q) we 
have considered here for the first time and is a part of the measure T(P\\Q). 

The one parametric generalization of the Kullback-Leibler ^7] relative in- 
formation studied in a different way by Cressie and Read 6j is given by 



;i.i4) $ s (p||q) 



E p s a, 
i=i 



s „1— s 



K S (P\\Q) = [s(s - I)]" 1 

K(Q\\P) = ±q t \n (j), 
i=i ' ' 

K{P\\Q) = tpM{f), 

for all P, Q G r n and s E (—oo, oo). 



The measure f!1.14|) admits the following particular cases: 

(i) $>- 1 {P\\Q) = \x 2 {Q\\P)- 

(ii) * Q {P\\Q) = K{Q\\P)- 
* 1/a (P||Q)=4[l-fl(P||g)] = 4/»(P||Q) 
^(P\\Q) = K(P\\Q). 



s = 

s = 1 



111 



IV 



^{p\\Q) = \x\p\\Q)- 



The measures B(P\\Q), h(P\\Q) and x 2 (P\\Q) appearing in parts (iii) and 
(v) above, are given by 

(1-15) B{P\\Q) = y/plqi, 
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(1.16) 



h(P\\Q) = l 



B{P\\Q) = ^£{yfc-y/$? 



and 



n 



(Pi 




(1.17) 



x 2 {P\\Q) 



E 



i=l 



respectively. 

The measure -B(P||Q) is famous as Bhattacharya [2] coefficient, the measure 
/i(P||Q) is famous as Hellinger [I"5] discrimination and the measures x 2 (-P||Q) 
is known by Chi-square [21] divergence. 

For more studies on the measure ()1.14|) refer to Liese and Vajda [TH], Taneja 
[2*7] . Taneja and Kumar [~~T] and Cerone et al. [""[. 

Our aim in this paper is to obtain bounds on the relative divergence mea- 
sures that we shall classify as non-symmetric divergence measures given by 
(ll.4|) - (jl.7|) and on the divergence measures classifying as symmetric divergence 
measures given by (jl.2j) . (jl.lOj) and (jl.llj) in terms of generalized relative in- 
formation or relative information of type s given by ()1.14j) . These bounds are 
studied by use of some properties of Csiszar [7j f-divergence. 

2. Csiszar /-Divergence 

Given a convex function / : [0, oo) — > M, the /—divergence measure intro- 
duced by Csiszar |7j is given by 



where P, Q G T n . 

It is well known in the literature [7J [S] that if f is convex and normalized, 
i.e., /(I) = 0, then the Csiszar function Cf(P\\Q) is nonnegative and convex 
in the pair of probability distribution (P, Q) eT n x T n . 

Now we shall prove the convexity and nonnegativity of the measures given 
in Section 1. 

Example 2.1. (Relative J- divergence). Let us consider 



in / TO) , then one gets C f {P\\Q) = D(P\\Q) := D x , where D (P\\Q) is as 



(2.1) 




(2.2) 




given by 
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Moreover, 

(2-3) f' Di ( X ) = ^ i+ ln fx + 1 

and 

x + 3 



(2-4) /£. 



.r 



z + 1 



T7ms /rom pO) u ' e see fb^ 00 ) > / or a ^ a: > ; one? hence, /^(x) is 
convex for all x > 0. v4/so ; we /lave /di(1) = 0. in mew of this we can say that 
the relative J- divergence is nonnegative and convex in the pair of probability 
distributions (P, Q) G T n x T n . 

Example 2.2. (Adjoint of relative J- divergence). Let us consider 

(2.5) f D2 (x) = (l-x) In {^pj , x G (0, oo) 

m / OHJ) . i/ien one oete C/(P||Q) = D(Q||P) := D 2 , where D(Q\\P) is as given 

by jod. 

Moreover, 

(2-6) ^(*) = + 1 

and 



x(x + 1) \ 2x 
3x + l 



x 



(2-7) /d v >, .. , 

x z [x + 1)- 

Thus from \2. l\j we see that fb 2 (x) > for all x > 0, and hence, fD 2 ( x ) ^ s 
convex for all x > 0. Also, we have /d 2 (1) =0. In view of this we can say 
that the adjoiont of relative J-divergence is nonnegative and convex in the pair 
of probability distributions (P,Q) G T n x T n . 

Example 2.3. (Relative J S- divergence). Let us consider 

(2.8) f Fl (x) = - x In ( ^-^j , x G (0, oo) 

in JjH)) ; then one gets C f (P\\Q) = F(P\\Q) := F 1 , where F (P\\Q) is as given 
by (El). 

Moreover, 

(2.9) /;,(•'•) ^TT-l^ ' 



,F ^ J ~2(x + l) V 2x 
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and 

Thus from \2.1fl) we see that fp^x) > for all x > 0, and hence, fF 1 (x) is 
convex for all x > 0. Also, we have f Fl {l) = 0. In view of this we can say that 
the relative JS-divergence is nonnegative and convex in the pair of probability 
distributions (P, Q) G Y n x Y n . 

Example 2.4. (Adjoint of relative JS-divergence). Let us consider 
(2.11) f F2(x) = ^-l n (^±l\ xG(0,oo) 

in PH)) . then one gets C f (P\\Q) = F(Q\\P) := F 2 , where F(Q\\P) is as given 
by ([13). 

Moreover, 

(2-12) f' F2 (x) 
and 



x 



2(x + l) 



(2.13) fp,(x) = 

JTras /rom we see that fp 2 (x) > for all x > 0, and hence, fF 2 (x) is 

convex for all x > 0. Also, we have /f 2 (1) = 0. In view of this we can say that 
the adjoint of relative JS-divergence is nonnegative and convex in the pair of 
probability distributions (P,Q) G Y n x Y n . 

Example 2.5. (Relative AG-divergence). Let us consider 

„ . . x — 1 x+l/x+l\ 

(2.14) /Gi ( X ) = __ + __ l n a;G(0,oo) 

m \2.1\) . then one gets Cf(P\\Q) = G(P\\Q) := G\, where G (P\\Q) is as given 
Moreover, 

(2.15) /iiW ^[^l + ln (£± 
and 

T/ifis /rom \2.1b}) we see that f'a^x) > /or all x > 0, and hence, fcAx) is 
convex for all x > 0. Also, we have /gi(1) — 0. In view of this we can say that 
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the relative AG-divergence is nonnegative and convex in the pair of probability 
distributions (P, Q) G T n x T n . 

Example 2.6. (Adjoint of relative AG- divergence). Let us consider 

(2-17) / G2 (^) = 1 T ^ + ^ln(^), xG(0,oo) 

in \2. then one gets Cf(P\\Q) = G(Q\\P) := G%, where G(Q\\P) is as given 
by^IM. 

Moreover, 

1 , fx + r 
2 

and 



(2.18) /g 2 (*) = :> 



(2-19) 



GaV ~ y 2(x + l) 

T/itts /rom \2.19) we see that fc 2 (x) > for all x > 0, and hence, fG 2 ( x ) ^ s 
convex for all x > 0. Also, we have /g 2 (1) = 0- I n view of this we can say that 
the adjoint of relative AG-divergence is nonnegative and convex in the pair of 
probability distributions (P,Q) G T n x T n . 

Example 2.7. (J-divergence). Let us consider 

(2.20) fj{x) = (x - 1) lnx, x G (0, oo) 

in \2.1)) . then one gets Cf(P\\Q) = J(P\\Q), where J (P\\Q) is as given by 

(El). 

Moreover, 

(2.21) f'j(x) = 1- x- 1 + \nx, 
and 

(2.22) m = ^±1 

T/ras /rom \2.22\) we see that f'J{x) > /or a// x > 0, and hence, fj(x) 
is convex for all x > 0. we nave /j(l) = 0. In view of this we can 

say that the J-divergence is nonnegative and convex in the pair of probability 
distributions (P, Q) G T n x T n . 

Example 2.8. (JS-divergence). Let us consider 

(2.23) fl ( x) = ^i nx -^i n (^±iy xG(0,oo) 

in \2.1)) . then one gets Cf(P\\Q) = I{P\\Q), where I (P\\Q) is as given by 



s 
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Moreover, 

(2.24) f'i.r) -\n f 



2 V 2x 

and 

(2.25) f»(x) - 



2x(x + 1) 

Thus from \2.2di) we see that f"(x) > for all x > 0, and hence, fi(x) 
is convex for all x > 0. Also, we have fi(l) = 0. In view of this we can 
say that the JS-divergence is nonnegative and convex in the pair of probability 
distributions (P,Q) G T n x T n . 

Example 2.9. (AG-divergence) . Let us consider 
(2.26) f T (x) = (^) In ,16(0,4 



2v^ 

in \2. then one gets Cf(P\\Q) = T(P\\Q), where T(P\\Q) is as given by 

cup. 

Moreover, 
(2.27) f T (x) = \ 

and 



1 - x" 1 + 2 In 



( x + l \ 
\2^J 



1 / T 2 _i_ 1 

(2.28) ./v'!.r) 



4 yx 3 + x 2 

T/ifis /rom iH.2f^ we see that f^(x) > for all x > 0, and hence, fr{x) 
is convex for all x > 0. Also, we have /t(1) = 0. In view of this we can 
say that the AG-divergence is nonnegative and convex in the pair of probability 
distributions (P, Q) G T n x T n . 

The above examples give only the nonnegativity and convexity of the sym- 
metric and non-symmetric divergence measures. Here we shall make use of 
this property to get bounds in terms of relative information of type s. For 
more properties of these measures refer to Taneja [HO] . 

3. Csiszar /-Divergence and Relative Information of Type s 
The following two theorems are due to Taneja j2Zj and Taneja and Kumar 

HQ. 
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Theorem 3.1. Let P,Q e T n and s£t:= (— oo, oo), t/ien we /jave 
(3.1) (K$ S (P||Q)<£*.(P||Q), 



(3.2) E*.(P\\Q) 



where 

E (ft - ft) In (a) , * = l" 

■ i=i ^ ' 

Let P,Q<E T n be such that there exists r, R with < r ^ ^ ^ R < oo, 
Vi G {1,2,..., n}, then 

(3.3) 0^$ s (P||Q) ^ A* a {r,R) 

where 



I K r s ^ 1 

(3.4) Aj, s (r,P) = ^(P-r) 2 J 7 ' 



^s — l_ r s — 1 

[ ?-r)(s 
l nfi-lnr 



S = 1 



Futher, if we suppose that 0<r^l^P< oo, r ^ R, then 

(3.5) 0^$ 8 (P\\Q)^Bi,(r,R) 
where 

(fl-l)(r'-l)+(l-r)(fl a -l) - ni 
(H-r)s(s-l) ' 6 7^ u > 1 

(3.6) ^(r,P)-< (fl - 1)ln r t ( r )ln ^ s = 



(R-r) 

(fl-l)r lnr+(l-r)fllnfl 
(fl-r) 

Moreover, the following inequalities hold: 
(3.7) £* s (P||Q)^ s (r,P), 



s = 1 



(3.8) B* s {r,R) ^A*,(r,R) 
and 

(3.9) B*.(r,i2)-$ fl (P||Q)^*.(r,i2). 

Theorem 3.2. Let / : J C ffi + — > K t/ie generating mapping is normalized, 
i.e., f(l) = and satisfy the assumptions: 

(i) /is twice differentiable on (r,R); 

(ii) t/iere exists rea/ constants m, M such that < m < M and 
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(3.10) m x 2 - s f"(x) ^ M, Vx G (r, R), -oo < s < oo. 
£/jen, we have 

(3.11) m$ s (P||Q) ^ C f (P\\Q) < M$ S (P||Q) 
and 

(3.12) m[ J B $s (P||Q)-^(P||g)]<E C/ (P|iQ)-C / (P||Q) 

<Af [£*.(P||Q)-$.(P||Q)]. 

Let P.Q g T„ be such that there exists r, R with < r ^ — ^ R < oo, 
Vi G {1, 2, n} ; then 

(3.13) m[A 0s (r,P) - $,(P||Q)] < A Cf (r,R) - C f (P\\Q) 

^M[A*.(r,R)-$ t (P\\Q)] 
Further, if we suppose that 0<r^l^P< oo, r ^ R, then 

(3.14) m[Bz a (r,R)-®s(P\\Q)}<B Cf (r,R)-C f {P\\Q) 

<M[P #s (r, R)-$ S (P\\Q)]. 

The Theorem l3.1l is obtained by applying some of the results due to Dragomir 
EH • The Theorem 13.21 unifies some of the results studied by Dragomir 
|11 | W] \ fTTT] . For an improved version of Theorem 13.21 refers to Taneja [2*%] . 

The aim here is to apply Theorem 13.21 by taking different values of / given 
by examples 12. 1H2.9I This we have done only applying the inequalities (j3.11|) . 
while the results for the inequalities ()3.12|) - (j3.14j) can be done on similar lines. 
These details are omitted here. 

4. Bounds On Non-Symmetric Divergence Measures 

In this section, we have applied the inequalities ()3.11j) and used the condition 
f!3.10|) to obtain bounds for the measures given in (jl.4}) - (jl.9|) . 



Theorem 4.1. The following bounds on relative J-divergence hold: 
(4-1) r ^^*s(P\\Q) < D(P\\Q) < ^^W llQ), s ^ 3 - 



and 

(4-2) ^'^yW llQ) < D{P\\Q) ^ r ^^ ^s{P\\Q), s > 2. 
Proof. Let us consider 



(4.3) g Dl (x) = x^f'^x) = X2 Z^J\ x G (0,oo), 
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where f'p (a?) is as given by (J2.4|) . 
From IfOjl . one can get 

(4.4) ^ (x) = _^!2li^Il x2 + ( 4s - 3 ) x + 3 ( s -2)] J>0, ^ 



i 



(x + 1) 3 1^0, s ^2 

In view of (14 .4|) . we conclude that 



(4.5) m = inf go^x) 



r 2 ~ s (r+3) < 3 

(r+1) 2 ' * 4 
R 2 - s (R+3) 



and 



(iM-1) 2 ' 
fl 2 ~ s (R+3) 



(R+1) 2 ' 6 ^ 4 



(4.6) M = sup g Dl (x)^, . {l 

*eM] [ (r+ \ )2 , s^2 

In view of (jOjl and (jSZTJ), we get the inequalities flUJ) and flO). □ 

Some particular cases of the Theorem 14.11 are summarized in the following 
corollary. 

Corollary 4.1. The following bounds hold: 

(4-7) $r±Vj?[Q\\ P ) <: D {P\\Q) < fi R ,\ 3 } x 2 (Q\\P), 



2 ( r+ 1)2^ y ^ v ^ 2(R+1) 



(4-8) *(Q||P) < D(P\\Q) < ^l^^gilP), 

(4.9) ^f^^ig)^(p|ig)<^9^MPiio) 

and 

(4.10) J^L x^HQ) < < ttP^xWIQ)- 



Proo/. Inequalities (j4~7|) . (|4~%)) and (JO)) follows from (|4~T|) by taking s = -1, 
s = and s = ~ respectively. The inequalities (|4.10|) follow form (|4.2|) by 
taking s = 2. □ 

The case s — 1 is not included in the inequalities (J4.1j) and (|4.2j) . This we 
shall do separately in the following proposition. 

Proposition 4.1. The following inequality hold: 
(4.H) D(P\\Q)^ 9 -K(P\\Q). 
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Proof. For s = 1 in (J4.3)) . we have 

(4.12) go(x) = ^±|. 
This gives 

(4.13) g D (x) = 



(x + l) 3 1 <C 0, x ^ 3 ' 

Thus we conclude from 1)4.13)1 that the function Qd{x) given by (J4.12)) is 
increasing in x G (0, 3) and decreasing in x G (3, oo), and hence 

(4.14) M = sup g D (x) = -. 

Now ()4.14|) together with 1)3.11)1 give the required result. □ 
Theorem 4.2. The following bounds on adjoint of relative J-divergence holds: 

(4-15) ^^yW llQ) < D(Q\\P) < ^^W llQ), s < -1 
and 

(416) fi (fl 3 +^ 1> $ ' (P|IQ) < B(QI|P) « *7r + iy ) *' (P|W) ' " ? i 



Proof. Let us consider 

= .t 2 - s ^ tvi = -, 

x + l 



(4.17) ^(x) = x 2 "7d» = ^Ti 3 . 3 ^ » * G (°'°°)' 



where /d 2 (x) is as given by ()2.7j) . 
From (jUTTjl, one can get 

x-*[3(s + l)x 2 + (4 g -l)x + s] f^Q, 

4.18) fl n j; = ; < , 

(x + 1) 3 |^0, s ^ i 

In view of (14.18)1 . we conclude that 

(4.19) m= mf g D {x) = mm g D (x) = i j^sj^+i) . i 

X6[r,fl] xe[r 5J R] [ ( ^ +1)2 , S ^ 4 

and 

(4.20) M = sup g D (x) = I r J s R +S) ' " r ■ 

xe[n-R] [ (r+i) 2 ' s ^ 4 
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Now from (ETTTfl) . (l4"2U|) and (jTTT| . we get the inequalities (ETTBT) and (pTTEjh 

□ 

Some particular cases of the Theorem 14.21 are summarized in the following 
corollary. 

Corollary 4.2. TTje following bounds hold: 

(4.21) !|L±i)^ ( Q||P) « D(Q\\P) < ^H+1)^(Q||P), 

and 

< 4 - 24 > mrrr^™ « D <«" p » « ^ttf^( p "«- 

Proof. Inequalities (|4.21j) follows from (J4.15|) by taking s = — 1. The inequal- 
ities (EPS) , (ft^ and flElD follows from (JO^ by taking s = §, s = 1 and 
s = 2 respectively. □ 

The case s = is not included in the inequalities ()4.15j) and (14.161) . This we 
shall do separately in the following proposition. 

Proposition 4.2. The following bound hold: 

(4.25) D(Q\\P)^~K(Q\\P), 
Proof. For s = in ()4.17j) . we have 

-I- 1 

(4.26) g D2 (x) ' 



(x + iy 

This gives 



(4.27) g'(x) 



3x-l j^O, x ^ i 



3 



(x + l) 3 K 0, X ^ | ' 

Thus from 1)4.27)1 we conclude that the function gD 2 ( x ) given by (J4.26)) is 
increasing in x e (0, |) and decreasing in x G (|, oo), and hence 

(4.28) M= sup to2 (x)= to2 (i) = ^. 

xg(0,oo) <J o 

Now ()4.28|) together with ()3.11|) give the required result. □ 
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Theorem 4.3. The following bounds on relative J S- divergence hold: 

(4.29) ^TW 2 ® sim) ^ F{m) ^ (^TIF $s(jP||g) ' s ^ ~ l 

and 

(4-30) _^_$ S (P||Q) ^ F (P\\Q) ^ -^J—<S> S ( P \\Q), 8 > 1. 



Proof. Let us consider 



„l-s 



(4.31) g Fl {x) = x 2 s f' Fi {x) = ^ + ^ 2 , x G (0, oo) 

where f'A(x) is as given by (|2.1()|) . 
From (IQll one can get 

(432) a'(x)~ ^^ + l)^ + (^-l)] f>0, s^-l 

In view of (14. 32(1 . we conclude that 



(4.33) m — mi g Fl (x 

and 



s^-1 



(r+1) 2 ' 

(Imp 



K 1 



(R+1) 2 ' 

„l-s 



S < -1 
s > 1 



(4.34) M = sup = 

xe[r,R] ^ (r+1) 2 ' 

Now ()4.33|) and ()4.34|) together with (j3.1H) give the required result. □ 

Some particular cases of the Theorem 14.31 are summarized in the following 
corollary. 

Corollary 4.3. The following bounds hold: 

(435) WTW*^ < F ^ Q) < wnrySMM 

(4 - 36) jithr K{pm) ^ F{PllQ) * VTTf K{m) 

and 

(437) 27vfTTF^( p ll« < F < p H« < 2^W^( p "«- 
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Proof. Inequalities (|4.35|) follows from (J4.29j) by taking s = — 1. The inequal- 
ities ()4.36|) and (J4.37|) follows from (j4.30|) by taking s = 1 and s = 2 respec- 
tively. □ 

The cases s = and s = ~ are not included in the inequalities ()4.29j) and 
(14.30)) . This we shall do separately in the following proposition. 

Proposition 4.3. The following bounds hold: 

(4.38) F(P\\Q)<:±K(Q\\P) 
and 

(4.39) F(P\\Q)^^-h(P\\Q). 



Proof. For s = in ()4.31|) . we have 
(4.40) g Fx {x) 



(x + l) 2 ' 

This gives 

x - 1 1^0, x < 1 



(4.41) g' Fi (x) 



(x+ l) 3 |< 0, x ^ 1 



From ()4.41|) we conclude that the function g Fl (x) given by (|4.41|) is increasing 
in i £ (0, 1) and decreasing in x G (1, oo), and hence 

(4.42) M= sup ^f 1 (x)=^ 1 (1) = -. 

xe(o,oo) 4 

Now (J4~42|) together with (j3~TT^ give the inequality (|4~38|) . 
Again let us take s = | in (|4.31j) . we have 



(4.43) 

This gives 

tAAA\ i / ^ 3x - 1 J ^ 0, X ^ \ 

(4-44) p' Pl (x) = -=-, — < 3 

V ; FlK ' 2^(x + lf \^ 0, x>§ 

Thus from ()4.44|) . we conclude that the function ^(x) given by (j4.43J) is 
increasing in x G (0, |) and decreasing in x G (|, oo), and hence 

1 3 /3 

(4.45) M= sup < 7Fl (x) = « ?Fl (-) = — r . 

xe(o,oo) <J 1(3 

Now (|P5Jl together with (j3~TTj) give the inequalities (jO^ . □ 
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Theorem 4.4. The following bounds on adjoint of relative JS-divergence hold: 

(4-46) ^^$ S (P||Q) ^ F (Q\\P) < _^_<&,(P||Q), s < 

and 

(4-47) ^^^(PUg) < F(Q\\P) < 8 > 2 

Proof. Let us consider 



2-s 



(4.48) ^ 2 (x) = x 2 "7f» = , ~j~~i~\2 i x G (0,oo) 

where f F (x) is as given by f)2. 131) . 
From (J4.48j) one can get 

In view of ()4.49j) . we conclude that 



(r+l) 2 



2, s^O 



(4.50) m = inf g F2 (xj - >, 2 
and 

(4.51) M= sup g*{x) = \$$" * ' 

xe[r,R] [ (r+1 )2 , S ^ 2 

Now ()4.50|) and ()4.51|) together with (j3.11|) give the required result. □ 

Some particular cases of the Theorem 14.41 are summarized in the following 
corollary. 

Corollary 4.4. The following bounds hold: 

(4-52) 2F^* 2(g||P) ^ F(Qm < 2Wl7 x2(g||P) ' 

(4.53) ^W 2KiQm ^ F(Qm ^ {^TW KiQm 

and 

(4-54) Tr^X 2 (P\\Q) < F(Q\\P) < ^y 2 X 2 (P\\Q), 
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Proof. Inequalities (|4.52j) and (|4.53|) follows from (J4.46j) by taking s — — 1 
and s = respectively. The inequalities ()4.54|) follow from ()4.47|) by taking 
s = 2. □ 

The cases s = \ and s = 1 are not included in the inequalities ()4.46j) and 
(I4.47J) . This we shall do separately in the following proposition. 

Proposition 4.4. The following bounds hold: 

(4.55) F(Q\\P)^^-h(P\\Q), 
and 

(4.56) F(Q\\P) < \k{P\\Q). 
Proof. For s = \ in ()4.48|) . we have 

(4.57) g F2 (x) 



(x + 1) 2 

This gives 

yfx(x — 3) J ^ 0, X ^ 3 



(4.58) g'(x) 



2(x+ l) 3 < 0, x ^ 3 



Thus from (|4.58|) we conclude that the function gF 2 ( x ) given by (|4.57|) is 
increasing in x G (0, 3) and decreasing in x G (3, oo), and hence 

(4.59) M= sup ^ F2 (x)=^ 2 (3) = — r . 

xe(o,oo) J-0 

Now ()4.59|) together with (|3.11|) give the inequality ()4.55|) . 
Again for s = in ()4.48|) . we have 

(4-60) ^ 2 ( x) = _^_ 

(x + 1)^ 

This gives 

x- 1 1^0, x ^ 1 



(4-61) 0k(*) 



(x + 1) 3 I <; 0, x ^ 1 



Thus from (|4.61|) . we conclude that the function gF 2 { x ) given by (j4.60p is 
increasing in x 6 (0, 1) and decreasing in x G (1, oo), and hence 

(4-62) M= sup g F2 ( x )=g F2 {l) = - 

xe(o,oo) 4 

Now (l47)2"J) together with (l3~TT| give the inequality (l4~57)J) . □ 
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Theorem 4.5. The following bounds on relative AG-divergence hold: 

( 4 -63) ^Tlf * (P||g) ^ G(P||(3) ^ 2fl«(fl+l) *' (P|IQ) ' S ^ _1 
and 

(4-64) ^1_ $,(P|1Q) < G(P\\Q) < r^+^$.(P||Q), s ^ 

Proof. Let us consider 

(4.65) <? Gl (x) = x 2 ~7 Gl (z) = ^f^y» * e (o- 00 ). 

where faAx) is as given by ()2.16j) . 
From ()4.65|) one gets 

, ar 1 "' [(s + l)s + s] j^O, s < -1 

(466) ^ (X) = WTTy — \<o, oo ■ 

In view of (14.66)1 . we conclude that 



i 



(4.67) m— inf g Gl (x 

and 



2r 3 (r+l) ' 



S < -1 



* " I 2R'(R+1) > S ^ 



S < -1 



(4.68) M= sup » Gl (s) = < . „ 

*e[r,fl] [ 2 r s (r+l)' S ^ U 

Now (|4.67)1 and ()4.68)) together with (j3.11j) give the required result. □ 

Some particular cases of the Theorem 14.51 are summarized in the following 
corollary. 

Corollary 4.5. The following bounds hold: 

< 469 > WTT/ mP) * G(P||q) * WPT) x2mn 

(470) 2(flTT) A ' (ol|p) * G{pm * 2(7TT) A ' (Qlln 

(471) 7m^) h(m) * G[Pm « t^™- 

(472) TO^W) < G < p H«' « 2KrTT) A '' p '' <3 '' 
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and 

(473) imkv/^ < G < p n« < ^ht) xHm) - 

Proof. Inequalities (|4.69j) follows from (J4.63j) by taking s — — 1. The inequali- 
ties (gZZDD , (l4~7TTl . (g7Z2D and (gZZ3) follows from (ETM1) by taking s = 0, s = |, 
s = 1 and s = 2 respectively □ 

Theorem 4.6. The following bounds on adjoint of relative AG-divergence 
holds: 

< 474 » 2^TT)*' (p l |0 ) < G W" P) < 27f|tT)*' (p|IO) - s < 1 

and 



Proof. Let us consider 



2- 

(4.76) <fe 2 (:r) = x 2 "7g» = , x E (0, oo), 

where fG 2 ( x ) is as given by (|2.19J1 . 
From ()4.76|) one gets 

U77) a' (x)- [(*-!)*+(*- 2)] /^0, *<1 

In view of f|4.77|) we conclude that 



(4.78) m= mi g G2 (x 

and 



2r s (r+l) : 



S < 1 



s < 1 
s > 2 



(4.79) M= sup g G2 (^)^<j 2 y i ): 

xe[r,fl] [2r s (r+l)' 

Now ()4.78|) and ()4.79|) together with (jH.llj) give the required result. □ 

Some particular cases of the Theorem 14.61 are summarized in the following 
corollary. 

Corollary 4.6. The following bounds hold: 



r 



3 



R 3 



(4-80) ^rrf^QWP) < G (QW p ) < AiRTTf ^W^ 
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(4.8i) ^ry^ll p ) < G (QW p ) ^ ^T) K ^ P ^ 



2r^/r 2R\/ R 

(4-82) ^£^h(P\\Q) < G(Q\\P) ^ ^±h(P\\Q), 

(4.83) -J^ K {JP\\Q) < G(Q\\P) < ^^y^^HQ) 

and 

(4-84) 4^Vl)X 2 (P||Q) < G(g||P) < ^pr/iPWQ)- 

Proof. Inequalities (g~HDj) , JHHIJ), fEEfl and follows from (jCTjl by tak- 

ing s = — 1, s = 0, s = 2 and s = 1 respectively. The inequalities (|4.84j) follows 
from (|4.75j) by taking s = 2. □ 

Remark 4 .1. T he inequalities UWj) , U^> , U^\> , WW , 

and \4-&4\l can be re-written as 

(4.85) r ^ £ t (P\ \Q) < R, t = 1, 2, 3, 4, 5, 6, 7 and 8, 

u>aere 



6(P||Q) = 
6(^1 IQ) = 
^(i^ilQ) = 

e 5 (p|ig) 
e 6 (p||g) 

^(^HQ) 



y/K{P\\Q)- JFVW) 



y/x*(P\\Q) - y/2F(Q\\P) 

*G(P\\Q) 
X 2 (Q\\P)-4G(P\\Qy 

_ K(Q\\P)-2G(P\\Q) 
2G(P\\Q) 

2G(Q\\P) 
" K(P\\Q)-2G(Q\\PY 
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and 

UPllQ) ~ 4G(Q\\P) ' 

respectively. 

5. Bounds on Symmetric Divergence Measures 

In this section we shall obtain bound on symmetric divergence measures 
given by (jl.2j) . (jl.K)j) and (jl.llj) in terms of relative information of type s 
given by ()1.14|) . Some particular cases are also given. 

Theorem 5.1. The following bounds on J-divergence hold: 

(5.1) 1±I$ S (P||Q) <C J(P||Q) ^ i±^$ s (p||Q), s <: 
and 

(5.2) !±?$,(P||Q) ^ J{P\\Q) ^ 1±T$ S (P||Q), s > 1 
Proof. Let us consider 

(5.3) &J (x) = x 2 ~ s f'J(x) = x~ s + x 1 " 5 = x e (0, oo), 

X s 

where f'j(x) is as given by (|2.22j) . 
From (|5.3jl one gets 

(5.4) <^(x) = x" 5 - 1 [(1 - s)x + (-s)} 
In view of ()5.4jh we conclude that 

(5.5) m = inf gj(x 
and 



l+r 
1+R 



^ 0, s ^ 
^0, s > 1 



s £ 



*eIr.JZ| I s ^ 1 



A' 1 



(5.6) m= sup ^)=r^ r s 5! 

xe[r,fl] [ 2 (l+r) ' S ^ 1 

Now (|5.5|) and ()5.6|) together with ()3.1H) give the required result. □ 

Some particular cases of the Theorem 15.11 are summarized in the following 
corollary. 
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Corollary 5.1. The following bounds hold: 

(5.7) r -±^ x \Q\\ P ) <c j(p\\q) <: ^±^! X 2 (Q || P)) 



(5.8) (l + r)K(Q\\P) ^ J(P\\Q) <: (1 + R)K(Q\ \P), 



(5.9) ±+JL K (P\\Q) ^ J{P\\Q) ^ l -^K{P\\Q) 

R r 

and 

(5-10) l -^x\P\\Q) ^ J(P\\Q) < l -7^x\P\\Q). 

Proof. Inequalities ()5.7|) and ()5.8|) follows from ((P| by taking s = — 1 and 
s = respectively. The inequalities (|5.9|) and (|5.10|) follows from (|5.2|) by 
taking s = 1 and s = 2 respectively. □ 

The cases s — | is not included in the inequalities (|5.1|) and (|5.2j) . This we 
shall do separately in the following proposition. 

Proposition 5.1. The following bound holds: 
(5.H) h(P\\Q)^lj(P\\Q). 



Proof. Take s = | in (|5.3)L we have 
(5.12) ^(x) ' 1 

and 

x - 1 I > 0, x > 1 



(5-13) <?j( 



x 



2x 3 / 2 < 0, x < 1 



Thus from (|5.13|) we conclude that the function gj(x) given by (|5.12|) is 

decreasing in x G (0, 1) and increasing in x G (1, oo), and hence 

(5.14) m= inf gj (x) = gj{l) = 2. 

a;G(0,oo) 

Now (|5.14|) together with (j3.11|) give the inequality (|5.11jl . □ 
Theorem 5.2. The following bounds on JS-divergence hold: 

(5-15) 2^TVf s{PllQ) ^ /(P||g) ^ 2^TRf siPllQ) ' S ^ ° 
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and 

(5.16) ^TT^ (jP||g) ^ /(p|l0) ^ 2(iTo $s(jP||g) ' s > L 

Proof. Let us consider 



(5.17) ffJ (x) = x 2 - s fl{x) = 7)7~~~~\~\ ' * e (0, oo), 

2(x + 1) 

where fj(x) is as given by (|2.25j) . 
From ()5.17|) one gets 

In view of f|5.18|) . we conclude that 

s < 



1-s 



(5.19) m = mf^ gi (x) = { 



and 



2(l+fi) ' 

,1-3 



s € 



(5.20) M= sup 9l (x)-- 

x€[r,.R] [ 2(l+r) ' s ^ 1 

Now ()5.19|) and ()5.2()|) together with (j3.11j) give the required result. □ 

Some particular cases of the Theorem 15.21 are summarized in the following 
corollary. 

Corollary 5.2. The following bounds hold: 

(5-21) ^rf 2 (QW p ) < m\Q) < i7^yx 2 (QI|P), 

(5-22) —I—j^P) < W?) < 20TR) K{Qlin 

(5-23) _^i_^lf(P||Q) < J(P||Q) ^ —L- K (P\\Q) t 

and 

(5-24) i(^yx 2 (P||Q) < i(P\\Q) < i(^yx 2 (P||g), 

Proof. Inequalities (|5.21|1 and ()5.22|) follows from (|5.15j) by taking s = — 1 and 
s = respectively. The inequalities ()5.23|) and ()5.24|) follows from (|5.16|) by 
taking s = 1 and s = 2 respectively. □ 
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The case s — | is not included in the inequalities (J5.15j) and (J5.16j) . This we 
shall do separately in the following proposition. 

Proposition 5.2. The following inequality hold: 

(5.25) I(P\\Q)^h(P\\Q). 

Proof. For s = ~ in ()5.17|) . we have 

'x 



(5-26) 9l (x) = - 

2(x + 1) 

and 

1 - x 1^0, ar < 1 



(5.27) g'j(x) 



A^/x(x + l) 2 | ^ 0, x^l' 



Thus from (|5.27j) we conclude that the function gi(x) given by (|5.26j) is 
increasing in x G (0, 1) and decreasing in x G (1, oo), and hence 

(5.28) M= sup gi (x)= gi (l) = ]. 

xe(o,oo) 4 

Now (l5~2"%J) and (pTTTJ) together give the inequality (|5~23j) . □ 

Theorem 5.3. The following bounds on AG-divergence hold: 

r - s fl + r 2 ) R~ s (l + R 2 ) 

(5-29) 4( \^ r) J ^HQ) < T(P\\Q) < 4( ; + + fl) * < "I 

and 

+ i? 2 ) r^fl + r 2 ) 

(5-30) 4( i + fl) S'CPIIQ) < r ( P ll^ < 4(i + r) ^(P\\Q), s > 2. 

Proof. Let us consider 

4(x 2 + x 3 ) 4(1 + 2;) 



(5.31) ^(s) = z 2 ~7t(z) = * + V = * G (0, 00) 



where /^(z) is as given by (|2.28jl . 

From ()5.31j) one gets 
(5.32) 

, x~ s ~ l [{s - l)x 3 + {s - 2)x 2 + {s + l)x + s] f^0, 

^ t(X) " 4(x + 1) 2 \<0, 2 

In view of (15. 32 J) , we conclude that 

^ S <-1 



(5.33) m= mfg T (x) = { R %+? ] 



di "* ^ ' I -R""(l+-R 2 ) \ o 

XelT ' R] 1 4(l+fl) S 
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and 

4(l+r) ' S ^ 2 

Now (J5.33)) and (J5.34)) together with (j3.11j) give the required result. □ 

Some particular cases of the Theorem 15.31 are summarized in the following 
corollary. 

Corollary 5.3. The following bounds hold: 
and 

< 536 > wrr 2 ™ < ™ < s^nV' p i'«- 

Proof. Inequalities ()5.35j) follows from (|5.29j) by taking s = — 1. The inequali- 
ties (I5.36j) follows from ()5.3Uj) by taking s = 2. □ 

The cases s — 0, s — ^ and s = 1 are not included in the inequalities (|5.29|) 
and ([5.30)1 . This we shall do separately in the following proposition. 

Proposition 5.3. The following inequalities hold: 
(5-37) ^lL K {Q\\P)^T(P\\Q), 



(5.38) ^\±K{P\\Q)^T{P\\Q) 
and 

(5.39) h(P\\Q)^T(P\\Q). 
Proof. Take s = in (|5.31j) . we have 

(5-40) = 0^ 

and 



(^ + 1-V2)(x+1 + V2) (>0, 
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In view of (I5.41j) we conclude that the function gr(x) given by (J5.4(J|) is 

decreasing in x G (0, y/2 - 1) and increasing in x G (a/2 — 1, oo), and hence 

(5.42) m= inf i ^) = M^-l) = 11 7^~. 

xe(o,oo) 4^2 2 

Now (|5.42|) together with (J3.ll)) give the inequality (|5.37|) . 
Again take s = 1 in (j5.31|) . we have 

< 5 - 43 > *■<*> - s£^r 

and 

(x-1-a/2)(x-1 + a/2) /^0, x^a/2 + 1 

(5.44) g T {x)- 4(a . + 1)a ^ 0> ^^2 + 1' 

In view of (15.44)) . we conclude that the function Qt(x) given by ()5.43)) is 

decreasing; in x G (0, y/2 + 1) and increasing in x G (\/2 + 1, oo), and hence 

/2 — 1 

(5.45) m= inf <? T (x) = g T {V2 + 1) = ^— — . 

a;G(0,oo) Z 

Now ()5.45|) together with (|3.11j) give the inequality ()5.38)) . 
Finally, take s = | in ()5.31jh we have 

< 5 - 46 > = tMtty 

and 

(x - 1) [{x - l) 2 + 6x] \ ^ 0, x > 1 



(5.47) ^(x) 



3x 3 / 2 (x + l) 2 1^0, x<l 



In view of (15.47)) we conclude that the function gr(x) given by (J5.46)) is 

decreasing in x G (0, 1) and increasing in x G (1, oo), and hence 

(5.48) m = inf g T (x) = g T (l) = - 

xe(o,oo) 4 

Now ()5.48)) together with (j3.ll)) give the inequalities ()5.39|) . □ 

Remark 5.1. (i) The inequalities appearing in \5. i5.ty) . \5.22]) and 
\5.2'J\i can be re-written as 

(5.49) r < ( t {P\\Q) < R, t= 1,2,3 and 4, 
where 

J(P\\Q) - K(Q\\P) 



b(P\\Q) 



K(Q\\P) 
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UP 
UP\\Q) 

and 

UP\\Q) 



K{P\\Q) 



J(P\\Q) - K(P\\QY 

K(Q\\P)-2I(P\\QY 
K(P\\Q)-2I(P\\Q) 



2I(P\\Q) 
respectively. 

(ii) In view of H5.ll]) . 115. 25)) and \5. 39\) . we have the following interesting 
relation 

(5.50) I(P\\Q) ^ h(P\\Q) ^ (t(P\\Q) or \j{P\\Q) 



A general form of the inequalities A5.5U\) can be seen in Taneja 
, where more kind of symmetric measures are also studied. 
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